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1.  Introduction 


1.1  Purpose  and  Overview 


The  U.S.  Army  Research  Laboratory  (ARL),  Human  Research  and  Engineering  Directorate 
(HRED)  conducted  the  second  in  a  series  of  human  factors  evaluations  of  the  OH-58F  Kiowa 
Warrior  Cockpit  and  Sensor  Upgrade  Program  (KW  CASUP)  crewstation  during  11-15  July 
2011  at  the  Systems  Simulation  and  Development  Directorate  (SSDD)  Apex  Laboratory, 
Redstone  Arsenal,  AL.  The  evaluation  was  conducted  to  identify  design  characteristics  of  the 
crewstation  that  enhanced  or  degraded  pilot  performance.  Aircrew  workload,  aircrew  situation 
awareness,  the  crewstation  interface,  pilot  visual  workload,  audio  alerting  system,  and  the 
potential  for  pilot  simulator  sickness  were  assessed.  The  OH-58F  crewstation  simulator  (figures 
1  and  2)  was  used  to  conduct  the  evaluation.  The  human  factors  evaluation  is  part  of  the 
continuous  assessment  process  to  develop  and  refine  the  crewstation  design.  The  continuous 
assessment  process  includes  modeling,  simulation,  developmental  and  operational  testing. 


Figure  1.  OH-58F  crewstation  simulator. 
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Figure  2.  OH-58F  instrument  panel. 

Pilots  received  1 .5  days  of  training  prior  to  the  beginning  of  the  evaluation.  The  training  was 
conducted  at  SSDD  and  consisted  of  classroom  instruction  and  hands-on  flight  training  using  a 
desktop  simulator  and  the  OH-58F  crewstation  simulator. 

The  pilots  flew  the  same  missions  during  training  that  they  later  flew  during  the  record  trials. 

The  mission  scenario  was  based  on  a  battlefield  environment  simulating  southwest  Asia.  Each 
successive  mission  increased  in  difficulty  in  order  to  impose  progressively  greater  workload  on 
the  pilots.  The  aircrews  performed  route,  area,  and  landing  zone/pick-up  zone  reconnaissance, 
call-for-fire,  and  specific  Aircrew  Training  Manual  (ATM)  tasks  during  each  mission.  Each 
ATM  task  has  prescribed  conditions  and  standards  to  which  both  crewmembers  had  to  perform 
to  ensure  mission  accomplishment.  The  pilots  rotated  seat  positions  during  the  evaluation. 

During  the  formal  evaluation,  two  sets  of  aircrews  conducted  three  reconnaissance  missions  and 
one  aircrew  performed  two  reconnaissance  missions  (for  a  total  of  eight  missions).  The  missions 
consisted  of  flight  segments  in  visual  meteorological  conditions  (VMC),  instrument 
meteorological  conditions  (IMC),  and  tactical  conditions.  The  mission  scenarios  were  developed 
by  the  Training  and  Doctrine  Command  (TRADOC)  Capability  Manager,  Reconnaissance 
Attack  (TCM  RA)  office,  Fort  Rucker,  AL.  The  scenarios  were  developed  in  accordance  with 
scout  aircraft  tactics,  techniques,  and  procedures  (TTP). 

Prior  to  each  mission,  the  pilots  received  a  briefing;  at  the  conclusion  of  each  mission,  the 
aircrew  completed  human  factors  surveys.  Before  and  after  each  mission,  they  completed  the 
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Simulator  Sickness  Questionnaire  (SSQ);  after  each  mission,  they  completed  the  Bedford 
Workload  Rating  Scale  (BWRS),  China  Lake  Situational  Awareness  (CLSA)  rating  scale,  and 
audio  alerting  survey.  The  pilots  completed  the  Pilot-Crewstation  Interface  (PCI)  questionnaire 
after  they  completed  all  of  their  missions.  During  each  mission,  pilots  wore  an  eye  tracker  that 
we  used  to  assess  their  visual  workload.  In  addition  to  the  pilot  data,  subject  matter  experts 
(SMEs)  provided  an  independent  assessment  of  aircrew  workload,  situation  awareness,  and 
mission  success.  After  each  mission,  the  SMEs  completed  an  aircrew  workload,  situation 
awareness  and  mission  success  survey.  After  the  aircrews  completed  the  mission  and  surveys, 
they  participated  in  a  mission  debriefing  and  after  action  review  (AAR). 

1.2  Assessment  of  Crew  Workload 

A  common  definition  of  pilot  workload  is  “the  integrated  mental  and  physical  effort  required  to 
satisfy  the  perceived  demands  of  a  specified  flight  task”  (Roscoe,  1985).  It  is  important  to  assess 
pilot  workload  because  mission  accomplishment  is  related  to  the  mental  and  physical  ability  of 
the  crew  to  effectively  perform  their  flight  and  mission  tasks.  If  one  or  both  pilots  experience 
excessively  high  workload  while  performing  flight  and  mission  tasks,  the  tasks  may  be 
performed  ineffectively  or  abandoned.  In  order  to  assess  whether  the  pilots  are  task-overloaded 
during  the  mission  profiles,  the  level  of  workload  for  each  pilot  must  be  evaluated. 

1.3  Bedford  Workload  Rating  Scale 

The  pilots  completed  the  BWRS  (appendix  A)  immediately  after  each  mission  to  rate  the  level  of 
workload  that  they  experienced  when  performing  ATM  tasks  during  missions.  Personnel  from 
the  TCM  RA,  ARL  HRED,  SSDD,  and  the  Armed  Scout  Helicopter  Program  Managers  Office 
selected  the  ATM  tasks  (appendix  A)  because  they  were  estimated  to  have  the  most  impact  on 
pilot  workload  during  the  missions. 

The  military,  civil,  and  commercial  aviation  communities  for  pilot  workload  estimation  (Roscoe 
&  Ellis,  1990)  have  used  the  BWRS  extensively.  It  requires  pilots  to  rate  the  level  of  workload 
associated  with  a  task  based  on  the  amount  of  spare  capacity  they  feel  they  have  to  perform 
additional  tasks.  Spare  workload  capacity  is  an  important  commodity  for  scout  pilots  because 
they  are  often  required  to  perform  several  tasks  concurrently.  For  example,  pilots  often  perform 
navigation  tasks,  communicate  via  multiple  radios,  monitor  aircraft  systems,  and  assist  the  pilot 
on  the  controls  with  flight  tasks  (e.g.,  maintain  airspace  surveillance)  within  the  same  time 
interval.  Mission  performance  is  reduced  if  pilots  are  task- saturated  and  have  little  or  no  spare 
capacity  to  perform  other  tasks.  Integration  of  the  OH-58F  crewstation  should  help  ensure  that 
pilots  can  maintain  adequate  spare  workload  capacity  while  performing  flight  and  mission  tasks. 
The  OH-58F  has  a  Capability  Development  Document  (CDD)  requirement  that  aircrew 
workload  not  exceed  6.0  (Threshold)  and  5.0  (Objective)  on  the  Bedford  Scale. 
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1.4  Visual  Workload 

An  eye  tracker  was  used  during  the  evaluation  to  assess  visual  gaze  and  dwell  times  for  the 
pilots.  The  data  were  collected  to  help  determine  how  well  the  design  of  the  crewstation  allowed 
the  flying  pilot  to  remain  focused  outside  the  aircraft  during  visual  flight  rules  (VFR)  flight  and 
how  well  the  non-flying  pilot  was  able  to  maintain  visual  focus  outside  the  aircraft  to  assist  with 
navigation  (e.g.,  identification  of  terrain  features),  local  security,  terrain  flight,  etc.  Visual  gaze 
and  dwell  time  data  help  identify  if  pilots  experience  excessive  visual  workload  or  cognitive 
capture  because  they  had  problems  interpreting  information  presented  to  them  on  the  crewstation 
displays. 

1.5  Assessment  of  Crew  Situation  Awareness 

Situational  awareness  (SA)  can  be  defined  as  the  pilot’s  mental  model  of  the  current  state  of  the 
flight  and  mission  environment.  A  more  fonnal  definition  is  “the  perception  of  the  elements  in 
the  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the 
projection  of  their  status  in  the  near  future”  (Endsley,  1988).  SA  was  important  to  assess 
because  it  had  a  direct  impact  on  pilot  and  system  performance.  Good  SA  should  increase  the 
probability  of  good  decision-making  and  performance  by  aircrews  when  conducting  flight  and 
mission  tasks  in  the  OH-58F. 

1.6  CLSA 

The  CLSA  (appendix  B)  is  a  unidimensional  rating  scale  for  pilots  to  report  their  perceived  SA. 
The  CLSA  uses  a  five  point  scale  that  requires  pilots  to  rate  their  knowledge  of  aircraft  energy 
state,  tactical  environment  and  mission,  ability  to  anticipate  and  accommodate  trends,  and  if  they 
shed  tasks  during  the  mission. 

1.7  Assessment  of  the  PCI 

The  PCI  directly  affects  crew  workload  and  SA  during  a  mission.  A  crewstation  designed  to 
augment  the  cognitive  and  physical  abilities  of  crews  will  minimize  workload,  enhance  SA,  and 
contribute  to  successful  mission  performance.  The  pilots  completed  a  PCI  questionnaire 
(appendix  C)  to  identify  any  problems  with  the  usability  of  the  controls,  displays,  or  subsystems. 

1.8  Assessment  of  Simulator  Sickness 

Simulator  sickness  has  been  defined  as  a  condition  where  pilots  suffer  physiological  discomfort 
in  the  simulator,  but  not  while  flying  the  actual  aircraft  (Kennedy  et  al.,  1989).  Generally, 
simulator  sickness  is  believed  to  be  caused  by  a  mismatch  either  between  the  visual  and 
vestibular  sources  of  information  about  self-motion,  or  between  the  sensory  information  (e.g., 
acceleration  cues)  presented  by  the  simulator  and  the  sensory  information  presented  by  the 
primary  aircraft  that  the  pilot  operates.  When  the  sensory  information  presented  by  the  simulator 
does  not  match  the  aircraft,  the  pilot’s  nervous  system  reacts  adversely  to  the  sensory  mismatch 
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and  the  pilot  begins  to  experience  discomfort.  Characteristics  of  simulator  sickness  include 
nausea,  dizziness,  drowsiness,  and  several  other  symptoms  (Kennedy  et  al.,  1989)  and  can  be 
distracting  to  pilots;  therefore,  assessment  of  simulator  sickness  is  important.  Pilot  distraction  is 
one  of  the  operational  consequences  of  simulator  sickness  listed  by  Crowley  (1987).  If  pilots  are 
distracted  by  the  discomfort  that  they  feel  during  missions,  their  performance  is  likely  to  suffer. 
Additionally,  the  discomfort  could  influence  the  perceived  levels  of  workload  and  SA  that  the 
pilots  experienced  during  a  mission. 

1.9  SSQ 

The  SSQ  was  administered  to  the  pilots  to  estimate  the  severity  of  physiological  discomfort  that 
they  experienced  during  missions  and  help  assess  whether  they  were  being  distracted  by  the 
discomfort.  The  SSQ  (Kennedy  et  al.,  1993)  is  a  checklist  of  16  symptoms  that  are  categorized 
into  three  subscales.  The  subscales  are  oculomotor  (e.g.,  eyestrain,  difficulty  focusing,  blurred 
vision),  disorientation  (e.g.,  dizziness,  vertigo),  and  nausea  (e.g.,  nausea,  increased  salivation, 
burping)  are  combined  to  produce  a  Total  Severity  score.  The  Total  Severity  score  is  an 
indicator  of  the  overall  discomfort  that  the  pilots  experienced  during  the  mission. 

1.10  Assessment  of  Audio  Alerting  System 

An  audio  alerting  system  should  immediately  alert  and  clearly  inform  the  aircrew  about  a 
problem  with  their  aircraft  that  requires  attention  (e.g.,  loss  of  an  engine).  Personnel  from 
Psycho-Linguistic  Research  Associates  (PLRA)  conducted  an  assessment  of  the  OH-58F  audio 
alerting  system.  The  purpose  was  to  assess  the  following  characteristics  of  the  alerting  system: 

•  Presentation  logic 

•  Composition  of  the  alerts 

•  Use  of  speech  and  sounds  for  the  various  alerts 

•  Utility  of  the  audio  alerting  system  functions  and  characteristics 

•  Workload  associated  with  using  the  system 

•  Pilots’  ratings  of  distinctiveness  and  speech  intelligibility  for  two  different  text-to-speech 
voices 

•  Pilots’  ratings  of  distinctiveness  and  recognizability  for  the  alerting  system  sounds 

1.11  SMEs 

Two  SMEs  typically  observed  the  missions  and  rated  crew  workload,  crew  SA,  and  mission 
success.  The  SMEs  provided  an  independent  assessment  of  the  workload  and  SA  levels 
experienced  by  the  crews  and  helped  determine  if  problems  with  crew  workload  or  crew  SA 
contributed  to  lack  of  mission  success. 
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The  SMEs  were  TCM  RA  personnel  who  had  substantial  experience  conducting  armed 
reconnaissance  and  attack  missions  and  were  familiar  with  the  OH-58F  crewstation.  They 
observed  each  mission  from  the  Battlemaster  station  where  they  could  observe  crewstation 
displays  and  the  out-the-window  view  provided  to  the  crew.  They  also  listened  to  all  audio 
communications  between  crewmembers  and  outside  sources  during  the  missions. 

1.12  Simulation  Environment 

SSDD  provides  modeling  and  simulation  support  of  weapon  systems  early  in  the  acquisition 
process.  This  is  accomplished  through  several  methods,  including  man-in-the-loop  simulators, 
distributed  simulation  experimentation,  and  constructive  simulation  development  in  the  SSDD 
Apex  Laboratory. 

The  Advanced  Prototype  Engineering  and  Experimentation  (APEX)  Lab  is  High-Level 
Architecture  (HLA)  and  Distributed  Interactive  Simulation  (DIS)  compliant,  and  has  the 
capability  to  connect  to  the  Army’s  Battle  Labs  and  other  distributed  simulation  facilities 
through  the  Defense  Research  and  Engineering  Network  (DREN). 

The  APEX  Lab  includes  a  Battlemaster  control  center  that  has  access  to  each  simulation  playing 
on  the  network  by  means  of  a  One  Semi-Automated  Forces  (OneSAF)  test  bed  terminal,  data 
collection  devices,  headset  communications,  and  video  monitoring.  All  exercises  are  controlled 
from  the  Battlemaster  station  to  ensure  that  all  players  are  engaged  in  the  exercise  and  all  data 
collection  devices  are  active.  Time  coordination  and  time  stamping  of  video  collection  devices 
is  achieved  through  an  integrated  Global  Positioning  System  (GPS)  clock.  Audio  and  video  are 
captured  and  routed  throughout  the  lab  and  various  conference  rooms  through  a  custom  video 
capture  and  switching  system. 

The  APEX  Lab  has  a  complete  synthetic  environment  development  team  that  is  able  to  develop 
custom,  correlated  terrain  databases  that  are  designed  to  specifically  enhance  realism  of  the 
immersive  environment  and  support  operational  scenarios  for  each  event.  The  Battlefield  Highly 
Immersive  Visual  Environment  (BHIVE)  provides  this  immersiveness  with  high-fidelity  out-the- 
window  (OTW)  terrain  databases  and  image  generators.  The  BHIVE  was  developed  in  support 
of  weapon  system  evaluation  in  an  HLA/DIS  compliant,  man-in-the-loop,  virtual  environment. 

It  was  designed  with  a  roll-in/roll-out  capability  to  allow  integration  of  several  types  of  devices 
into  the  environment  through  a  standard  interface.  This  provides  the  flexibility  to  immerse 
multiple  types  of  cockpits  in  a  realistic  and  reusable  synthetic  world.  Six  projectors  are  used  to 
project  the  OTW  view  onto  an  180°x60°  directional  curved  dome.  The  projection  system  is 
capable  of  edge  blending  for  high-definition  synthetic  environments. 
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2.  Method 


2.1  Participants 

Participants  were  five  pilots  from  Fort  Rucker,  AL,  and  one  pilot  from  the  Tennessee  Army 
National  Guard.  The  pilots  were  warrant  officers  and  rated  in  the  OH-58D.  One  pilot  held  the 
rank  of  CW2,  four  pilots  held  the  rank  of  CW3,  and  one  pilot  held  the  rank  of  CW4.  Two  pilots 
were  assigned  to  the  TCM  RA,  one  pilot  was  assigned  to  the  1 10th  Aviation  Brigade,  one  pilot 
was  assigned  to  the  Directorate  of  Evaluation  and  Standardization,  one  pilot  was  assigned  to  the 
Directorate  of  Training  and  Doctrine,  and  one  pilot  was  assigned  to  the  1-230  Air  Cavalry 
Squadron,  Tennessee.  The  pilots  represented  a  fairly  broad  range  of  experience  with  their  total 
flight  hours  ranging  from  800-5300  hours.  The  relevant  demographic  characteristics  of  the 
pilots  are  listed  in  table  1. 

Table  1.  Pilot  demographics  (N  =  6). 


Summary  of 
demographic 
Characteristics 

Age 

(yrs) 

Flight  hours 
in  OH-58D 

Total  flight  hours 
in  Army  aircraft 

Mean 

37.5 

2510 

2868 

Median 

37.0 

2525 

2750 

Range 

35^2 

600-4200 

800-5300 

2.2  Data  Collection 

The  pilot  questionnaires  were  developed  in  accordance  with  published  guidelines  for  proper 
format  and  content  (O’Brien  and  Charlton,  1996).  A  pretest  was  conducted  to  refine  the 
questionnaires  and  to  ensure  that  they  could  be  easily  understood  and  completed  by  pilots  and 
SMEs. 

The  pilots  completed  the  workload,  audio  alerting,  and  SA  questionnaires  after  each  mission. 
They  completed  the  SSQ  before  and  after  each  mission.  The  pilots  completed  the  PCI 
questionnaire  after  they  completed  all  of  their  missions.  The  SMEs  completed  questionnaires 
after  each  mission.  Additional  data  were  obtained  from  the  pilots  and  the  SME  members  during 
post-mission  discussions  and  the  final  AAR.  Questionnaire  results  were  clarified  with 
information  obtained  during  post-mission  discussions  and  the  daily  AARs. 

2.3  Eye  Tracker  System 

Pilot  visual  gaze  and  dwell  times  were  collected  with  an  eye  tracking  system  (Model  501)  from 
Applied  Science  Laboratories  (ASL)  and  a  head  tracking  system  (Polaris  Spectra)  from  Northern 
Digital  Incorporated  (NDI).  These  systems  were  used  because  they  allowed  unrestricted  head 
movement  during  data  collection  and  were  compatible  with  the  HGU-56  flight  helmet  worn  by 
the  pilots.  This  technology  allowed  the  collection  of  digital  data  that  specified  point  of  gaze  with 
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respect  to  stationary  objects  within  the  crewstation.  The  ASL  software  allowed  data  collectors  to 
continuously  monitor  the  eye  position  of  the  pilots  by  crosshairs  superimposed  over  live  imagery 
(figure  3).  The  software  also  included  a  built-in  analysis  tool  that  allowed  data  to  be  viewed  in 
tabular  or  graphical  format. 


Figure  3.  Eye  tracker  scene  camera  monitors  and  control  panel  interface. 

2.4  Data  Analysis 

Pilot  responses  to  the  BWRS,  CLSA,  SSQ,  audio  alerting,  and  PCI  questionnaires  were  analyzed 
with  means  and  percentages.  The  eye  tracker  data  were  summarized  by  calculating  the  total 
percentage  of  fixations  that  occurred  for  the  different  areas  of  interest  (AOI).  Eight  AOIs  were 
created  for  the  pilot  and  copilot: 

•  5x7  multifunction  display  (MFD) 

•  6x8  MFD 

•  Kneeboard 

•  OTW 

•  Outer  left  instrument  panel  (copilot) 

•  FPCAP  (pilot) 

•  Fower  console 
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A  final  category,  called  “Other,”  captured  eye  fixations  not  focused  on  a  specific  AOI.  Results 
of  the  eye  tracker  data  are  contained  in  figures  5  and  6. 

2.5  Evaluation  Limitations 

Limitations  included  the  small  sample  size  of  pilots  (N=6)  who  participated  in  the  crewstation 
simulation  assessment,  limited  amount  of  training  provided  to  the  pilots,  and  hardware/software 
limitations.  The  primary  hardware  and  software  limitations  follow: 

•  Built-In  Test  (BIT)  functions  were  not  available 

•  Hands-On  Grip  (HOG)  was  functional  for  Joint  Variable  Message  Format  (JVMF)  free  text 
only 

•  Video  Tape  Recorder  (VTR)  was  nonfunctional 

•  Health  Usage  Monitoring  System  (HUMS)  was  not  available 

•  Circuit  Breaker  Panels  were  nonfunctional 

•  Nose  Mounted  Sensor  (NMS)  Linear  Motion  Control  did  not  function  if  the  NMS  was 
pointed  off  the  ground 

•  Video  rocker  switches  were  nonfunctional 

•  Hellfire  was  functional  in  lock-on-before-launch  (LOBL)  mode  only 

•  Pulse  Interval  Module  (PIM)  codes  were  not  available 

•  Certain  display  font  sizes  were  slightly  inaccurate 

•  Satellite  Demand  Assigned  Multiple  Access  (DAMA)  and  frequency  hop  were 
nonfunctional 

•  Display  map  did  not  have  Controlled  Image  Base  (CIB)  and  12.5  software  functionality 

These  limitations  are  fairly  common  when  simulating  a  complex  aviation  system.  However,  the 
information  and  data  listed  in  the  Results  and  Summary  sections  of  this  report  should  be 
interpreted  based  on  these  limitations.  Additional  data  should  be  collected  during  future 
simulations  and  tests  to  augment  and  expand  the  findings  contained  in  this  report. 
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3.  Results 


3.1  Crew  Workload 

3.1.1  Mean  Workload  Ratings  for  Flight  and  Mission  Tasks 

The  average  mission  workload  rating  was  2.75  for  the  pilot  and  3.38  for  the  copilot  (figure  4). 
These  ratings  indicate  that  the  pilots  and  copilots  typically  felt  that  workload  was  tolerable  for 
the  flight  and  mission  tasks  they  performed  during  the  missions.  Additionally,  the  ratings 
indicate  that  the  pilots  and  copilots  typically  felt  they  had  enough  spare  workload  capacity  to 
perform  all  desirable  additional  tasks  (within  the  same  time  interval)  during  missions. 


The  average  workload  ratings  for  flight  and  mission  tasks  (appendix  A)  provided  by  the  pilots 
and  copilots  were  lower  than  the  Objective  (5.0)  and  Threshold  (6.0)  BWRS  workload  rating 
requirements  contained  in  the  OH-58F  CDD  except  for  the  following  tasks: 

•  Perform  Emergency  GPS  Recovery  Procedure 

•  Perform  an  Autorotation 

•  Perform  Unusual  Attitude  Recovery 

•  Respond  to  IMC  Conditions 

•  Call  for  Tactical  Air  Strike 
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The  pilots  commented  that  the  higher  workload  they  experienced  during  these  tasks  was  due  to 
the  intensive  nature  of  the  tasks  and  different  flight  handling  characteristics  of  the  simulator 
(simulator  flight  model)  versus  the  actual  aircraft. 

3.1.1  SME  Workload  Ratings 

SMEs  provided  an  overall  Bedford  workload  rating  for  each  pilot  during  each  mission  that  they 
observed.  The  average  SME  Bedford  workload  rating  (table  2)  was  3.31  for  the  pilot  and  3.46 
for  the  copilot  for  all  missions,  which  indicates  the  SMEs  believed  that  workload  was  tolerable 
for  the  pilots  and  copilots.  Additionally,  the  ratings  indicate  that  the  SMEs  felt  that  the  pilots 
and  copilots  typically  had  sufficient  spare  workload  capacity  for  attention  to  additional  tasks. 
SME  workload  comments  are  listed  in  appendix  A. 


Table  2.  SME  Bedford  workload  ratings. 


Pilot 

Copilot 

Overall 
(L+R  seat) 

Average  Workload 

3.31 

3.46 

3.39 

Standard  Deviation 

0.75 

0.78 

— 

3.1.2  Visual  Workload 

Figure  5  shows  the  percentage  of  time  that  the  pilots  were  visually  focused  (during  VFR  flight) 
on  each  AOI  during  the  missions.  It  is  interesting  to  note  that  the  copilots  typically  spent  only 
14%  of  the  time  visually  focused  OTW  during  missions.  The  copilot  needs  to  periodically 
maintain  visual  focus  outside  the  aircraft  to  assist  with  navigation  (e.g.,  identification  of  terrain 
and  cultural  features)  and  airspace  surveillance.  While  the  percentage  of  time  that  the  copilot 
maintained  visual  focus  outside  the  aircraft  was  higher  than  in  the  previous  OH-58F  crewstation 
assessment  (table  3),  maintaining  visual  focus  outside  the  aircraft  for  only  14%  of  a  typical  zone 
reconnaissance  mission  may  be  too  low  to  adequately  assist  the  pilot  with  crew  tasks  such  as 
obstacle  avoidance  and  terrain  flight  navigation.  The  low  percentage  of  time  that  the  copilots 
were  visually  focused  outside  the  aircraft  was  likely  due  to  the  workload  required  to  manage 
information  on  the  crewstation  displays,  operate  the  NMS  and  the  lack  of  in-depth  experience 
that  the  copilots  had  with  the  crewstation  interface.  It  will  be  important  to  assess  visual  gaze 
during  future  human  factors  engineering  (HFE)  crewstation  evaluations  to  determine  if  the 
copilot  workload  precludes  maintaining  adequate  visual  focus  outside  the  aircraft. 
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Pilot  Visual  Gaze  Distribution  Copilot  Visual  Gaze  Distribution 


Figure  5.  Pilot  and  copilot  visual  gaze  and  dwell  times. 

Table  3.  Comparison  of  eye  tracker  results  for  OH-58F,  AH-64D,  and  ARH  simulations. 


AH-64D/UAS 
Workload 
Assessment 
(Block  III) 

AH-64D 
Workload 
Assessment 
(Block  III) 

ARH  HFE- 
CAAS 
Evaluation 

OH-58F  HFE  #1 
Evaluation 

OH-58F  HFE  #2 
Evaluation 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

CPG 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Outside 

75% 

4% 

75% 

6% 

75% 

3% 

61% 

7% 

50% 

14% 

Inside 

25% 

96% 

25% 

94% 

25% 

97% 

39% 

93% 

50% 

86% 

The  pilots  typically  spent  50%  of  the  time  visually  focused  OTW  during  VFR  missions.  The 
amount  of  time  (50%)  that  the  pilots  were  visually  focused  inside  the  aircraft  was  due  to 
instrument  scans,  lack  of  in-depth  experience  with  the  crewstation  interface,  and  the  “helping 
behaviors”  of  the  pilots  when  flying  the  aircraft.  The  pilot  occasionally  helped  the  copilot 
manage  information  on  the  crewstation  displays,  which  kept  both  of  them  visually  focused  inside 
the  crewstation. 

Comparison  of  Eye  Tracker  Data 

Table  3  shows  a  comparison  of  OH-58F,  AH-64D,  and  Armed  Reconnaissance  Flelicopter 
(ARH)  eye  tracker  data  for  VFR  flight  during  simulations.  While  the  simulator,  missions,  and 
personnel  experience  levels  were  different  for  each  simulation  evaluation,  it  is  interesting  to  note 
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the  differences  in  visual  gaze  and  dwell  times  for  each  evaluation.  Past  discussions  with  several 
AH-64D  and  OH-58D  pilots  identified  that: 

1.  The  flying  pilot  should  typically  maintain  visual  gaze  and  dwell  times  outside  the  aircraft 
for  70-80%  of  the  time  during  VFR  flight. 

2.  The  non-flying  pilot  should  maintain  visual  gaze  and  dwell  times  outside  the  aircraft  for 
30-40%  of  the  time  during  VFR  flight. 

3.2  Crew  SA 

3.2.1  Pilot  SA  Ratings 

The  pilots  and  copilots  reported  that  they  typically  experienced  adequate  to  good  SA  during 
missions;  they  were  able  to  maintain  knowledge  of  the  aircraft  energy  state,  tactical  environment, 
and  mission;  were  able  to  partially  anticipate  and  accommodate  trends;  and  had  minimal  task 
shedding  (due  to  high-workload)  during  the  missions.  The  pilots  and  copilots  typically  reported 
“fairly  high”  levels  of  SA  of  the  location  of  their  ownship,  route  information  (e.g.,  phase  lines), 
friendly  units,  and  status  of  the  aircraft  systems  (e.g.,  fuel  consumption).  They  reported  “fairly 
high”  to  “borderline”  SA  of  the  location  of  enemy  units  and  location  of  cultural  features  (e.g., 
bridges). 

3.2.2  SME  SA  Ratings 

The  SMEs  provided  an  independent  assessment  of  SA  based  on  the  scale  shown  in  table  4.  The 
mean  SME  SA  rating  was  2.54.  This  indicates  that  the  SMEs  reported  that  the  crews  typically 
had  adequate  levels  of  SA  with  some  variation  between  aircrew  perception  of  entities  on  the 
battlefield  and  reality.  The  SMEs  commented  that  crew  fixation  on  the  NMS  imagery  presented 
on  the  MFD  and  using  too  narrow  of  a  zoom  on  the  NMS  reduced  overall  pilot  SA  during  four 
missions. 

Table  4.  SME  SA  rating. 


Mean  Rating 
2.54 

(SD  =  1.13) 


SME  SA  Ratings 

1  Crew  was  consistently  aware  of  all  entities  on  the  battlefield. 

2  Crew  was  aware  of  the  battlefield  with  minor  or  insignificant 
variation  between  perception  and  reality. 


^  Crew  was  aware  of  the  battlefield.  Variation  between  reality 
and  perception  did  not  significantly  impact  mission  success. 

^  SA  needs  improvement.  Lack  of  SA  had  some  negative  effect 
on  the  success  of  the  mission. 

5  Lack  of  SA  caused  mission  failure. 
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3.3  SME  Mission  Success  Ratings 


At  the  end  of  each  mission,  SMEs  rated  whether  the  mission  was  a  success  or  failure.  The 
criteria  used  by  the  SME  to  rate  mission  success  or  failure  was  whether  the  aircrew  completed 
the  mission  requirements  and  did  not  get  shot  down  or  crash.  The  SMEs  rated  all  of  the  missions 
(100%)  as  “successful”  and  “objectives  completed”  (figure  6). 


Figure  6.  SME  ratings  of  mission  success. 


3.4  PCI 

The  pilots  were  mostly  favorable  in  their  ratings  of  the  crewstation  interface  (appendix  C).  They 
reported  they  were  able  to  effectively  use  the  MFD  pages  and  functions;  quickly  navigate 
through  the  pages,  sub-pages,  and  overlays  on  the  crewstation  displays;  easily  use  the  switches 
on  the  cyclic  and  collective  and  easily  use  the  switches  to  control  the  NMS.  They  also  reported 
that  it  was  easy  to  detect  the  Warnings,  Cautions,  and  Advisories  on  the  MFD  and  entry  into 
operational  limits.  Three  pilots  commented  that  the  colors  used  for  the  symbology  (e.g., 
magenta)  on  the  MFDs  made  it  difficult  for  them  to  quickly  and  easily  distinguish  the 
symbology.  The  pilots  reported  that  the  Control  Display  Subsystem  (CDS)  5  software  was 
somewhat  quicker  and  easier  to  use  than  the  CDS  4  software.  They  also  reported  that  the  6x8 
display  enhanced  SA  and  that  NMS  functionality  is  an  improvement  over  the  Mast  Mounted 
Sight  (MMS)  on  the  OH-58D  Kiowa  Warrior.  Several  minor  crewstation  improvements  that 
should  be  made  to  the  OH-58F  were  reported  by  the  pilots  (appendix  D). 

3.5  Simulator  Sickness 
3.5.1  OH-58F  Simulator 

The  pilots  and  copilots  reported  that  they  typically  experienced  very  mild  to  mild  simulator 
sickness  symptoms  during  the  evaluation.  The  overall  mean  Total  Severity  (TS)  score  (post 
mission)  for  the  pilots  and  copilots  was  19.23  (table  5).  The  mean  TS  score  for  the  pilot  was 
18.17  and  the  mean  TS  score  for  the  copilot  was  20.30.  When  flying  the  aircraft,  the  pilots  were 
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visually  immersed  in  the  changing  scene  outside  the  aircraft  and  often  transitioned  their  visual 
gaze  inside  the  aircraft  to  monitor  information  displayed  on  the  MFDs.  The  copilots  primarily 
maintained  their  visual  gaze  inside  the  aircraft  to  monitor  and  input  data  into  their  MFDs.  One 
pilot  commented  that  visual  after-effects  increased  about  30  min  after  the  end  of  the  missions 
that  he  performed;  all  pilots  commented  that  the  simulator  sickness  symptoms  were  typically 
mild.  Overall,  the  simulator  did  not  appear  to  induce  debilitating  simulator  sickness  symptoms 
and  should  continue  to  be  a  suitable  simulation  environment  for  future  assessments. 

Table  5.  SSQ  ratings. 


Condition 

Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

TS  Score  (Mean) 

Pre-Mission  Pilot 

10.90 

17.33 

0 

12.82 

Pre-Mission  Copilot 

2.73 

11.91 

1.99 

7.48 

Post  Mission  Pilot 

10.90 

19.90 

13.92 

18.17 

Post  Mission  Copilot 

6.81 

22.74 

23.86 

20.30 

Post  Mission  Combined 
(Pilot  and  Copilot) 

8.86 

21.32 

18.91 

19.23 

3.5.2  Comparison  of  OH-58F  Simulator  SSQ  Scores  to  Other  Helicopter  Simulators 

To  assess  whether  the  SSQ  ratings  provided  by  the  pilots  during  the  HFE  assessment  were 
similar  or  different  to  ratings  obtained  in  other  helicopter  simulators,  the  mean  SSQ  scores  for 
the  OH-58F  simulator  were  compared  to  the  mean  SSQ  scores  for  several  other  helicopter 
simulators  (table  6).  The  other  helicopter  simulators  were  the  AH-64D,  S-3H,  and  CH-46E, 

CH  53D,  CH-53F,  Sikorsky  RAH-66  Engineering  Development  Simulator  (EDS),  RAH-66 
Comanche  Portable  Cockpit  (CPC),  Armed  Reconnaissance  Helicopter  (ARH),  and  the  simulator 
used  during  the  UH-60M  from  the  Early  User  Demo  (EUD)  and  Limited  Early  User  Evaluation 
(LEUE).  These  simulators  typically  induced  very  mild  to  mild  simulator  sickness  symptoms  in 
pilots. 

Table  6.  Comparison  of  OH-58F  simulator  SSQ  ratings  with  other  helicopter  simulators. 


Simulator 

Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

Total  Severity  Score 
(Mean ) 

ARH  Simulator 

18.02 

21.48 

9.28 

20.15 

OH-58F  Simulation  #2 

8.86 

21.32 

18.91 

19.23 

SH-3H 

14.70 

20.00 

12.40 

18.80 

OH-58F  Simulation  #1 

16.43 

12.21 

10.05 

15.16 

RAH-66  EDS 

11.84 

14.98 

4.54 

13.25 

CH-53F 

7.50 

10.50 

7.40 

10.00 

RAH-66  CPC 

3.29 

12.94 

7.89 

9.80 

UH-60M  (LEUE) 

6.36 

11.81 

3.09 

9.15 

AH-64D  -  IUAS  (RACRS) 

9.01 

7.58 

4.64 

8.51 

UH-60M  (EUD) 

13.88 

6.89 

0 

8.50 

CH-53D 

7.20 

7.20 

4.00 

7.50 

CH-46E 

5.40 

7.80 

4.50 

7.00 

AH-64D  VUIT-2  (RACRS) 

3.18 

5.05 

4.64 

4.98 
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3.6  Audio  Alerting 

The  overall  design  of  the  audio  alerting  system  received  good  to  high  ratings  from  the  pilots  for 
alert  presentation  logic,  the  utility  of  the  audio  alerting  system  functions  and  characteristics,  the 
audio  workload  associated  with  using  the  system,  and  pilots’  reported  air  vehicle  and  tactical  SA 
achieved  during  the  simulated  missions.  Two  design  features  received  moderate  utility  ratings. 
There  were  no  differences  in  the  pilots’  ratings  of  the  two  text-to-speech  voices  for 
distinctiveness.  The  Version  A  voice  was  initially  rated  as  more  intelligible  than  the  Version  B 
voice,  but  this  difference  diminished  then  disappeared  as  the  pilots  progressed  from  hearing  the 
audio  alerts  in  isolation  to  hearing  them  during  practical  cockpit  exercises  to  hearing  them  in 
mission  context.  The  pilots’  ratings  and  their  comments  did  indicate  a  strong  preference  for  the 
current  audio  alerting  tones  implemented  in  the  OH-58D  aircraft  and  a  bias  against  any 
mechanical  sounding  voice,  due  to  negative  associations  with  the  APR-39A  radar  warning 
system  in  their  current  aircraft.  Based  on  the  results  discussed  in  the  audio  alerting  assessment, 
PLRA  made  the  following  recommendation. 

The  current  Version  A  voice  alerts  that  are  loaded  in  the  DICS  should  be  replaced  with  one  of 
the  following: 

•  A  new  set  of  alerts  spoken  in  a  more  natural  sounding  but  still  highly  distinctive  and  highly 
intelligible  female  voice 

•  A  new  set  of  alerts  spoken  in  the  Version  A  voice  with  more  consistent  voice  qualities  and 
speaking  rates  across  all  the  alerts 

•  The  Version  A  alerts  as  tested  in  HFE#2  with  some  changes 

3.7  Crewstation  Design  Enhancements 

Several  design  enhancements  to  the  OH-58F  crewstation  displays  have  been  made  via  the 
Crewstation  Working  Group  and  simulation  process  (see  figure  7).  These  enhancements 
improved  the  functionality  and  presentation  of  display  pages  to  pilots  and  overall  crewstation 
interface.  Examples  of  the  enhancements  are  improved  color-coding  of  battlefield  graphics, 
reduced  number  of  button  presses  to  display  information  on  MFDs,  refinement  of  the  composite 
map  page,  and  enhanced  presentation  of  operational  limits  on  the  Systems  Page.  Additionally, 
the  Crewstation  Working  Group  and  simulation  process  has  aided  in  the  refinement  of  TTP  for 
OH-58F  operational  employment. 
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Added  capability  to  view  additional 

information  on  map  display  _ 

(e  g  ,  operational  limits)  to  increase 
pilot  situational  awareness 


Changed  bezel  button  logic  for 
map  and  sensor  display  modes 
to  enhance  usability 


l.'». 

Figure  7.  Examples  of  crewstation  design  enhancements. 


4.  Summary 


4.1  Crew  Workload 

The  pilots  and  copilots  reported  that  workload  was  manageable  for  the  flight  and  mission  tasks 
they  performed  during  the  missions  and  the  SMEs  also  reported  that  workload  was  manageable 
for  the  pilots  and  copilots.  The  average  mission  workload  ratings  provided  by  the  pilots, 
copilots,  and  SMEs  were  lower  than  the  Objective  (5.0)  and  Threshold  (6.0)  BWRS  workload 
rating  requirements  contained  in  the  OH-58F  CDD. 

4.2  Visual  Workload 

The  copilots  spent  14%  of  the  time  visually  focused  OTW  during  VFR  missions.  The  small 
percentage  of  time  that  the  copilots  were  visually  focused  outside  the  aircraft  was  likely  due  to 
the  workload  required  to  manage  information  on  the  crewstation  displays,  operate  the  NMS,  and 
the  lack  of  in-depth  experience  that  the  copilots  had  with  the  crewstation  interface.  The  pilots 
typically  spent  50%  of  the  time  visually  focused  OTW  during  VFR  missions.  The  amount  of 
time  (50%)  that  the  pilots  were  visually  focused  inside  the  aircraft  was  due  to  instrument  scans, 
lack  of  in-depth  experience  with  the  crewstation  interface,  and  the  “helping  behaviors”  of  the 
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pilots  when  flying  the  aircraft.  The  pilot  occasionally  helped  the  copilot  manage  information  on 
the  crewstation  displays,  which  kept  both  of  them  visually  focused  inside  the  crewstation. 

4.3  Crew  SA 

The  pilots  and  copilots  reported  that  they  typically  experienced  adequate  to  good  SA  during 
missions,  were  able  to  maintain  knowledge  of  the  aircraft  energy  state,  tactical  environment  and 
mission;  were  able  to  partially  anticipate  and  accommodate  trends;  and  had  minimal  task 
shedding  (due  to  high-workload)  during  the  missions.  They  reported  that  they  had  adequate  SA 
of  most  battlefield  elements  (e.g.,  location  of  their  ownship,  route  information)  during  the 
missions.  The  SMEs  reported  that  the  crews  typically  had  adequate  levels  of  SA  during 
missions. 

4.4  Pilot-Crewstation  Interface 

The  pilots  reported  they  were  able  to  effectively  use  the  MFD  pages  and  functions;  quickly 
navigate  through  the  pages,  sub-pages,  and  overlays  on  the  crewstation  displays;  easily  use  the 
switches  on  the  cyclic  and  collective;  and  easily  use  the  switches  to  control  the  NMS.  They  also 
reported  that  it  was  easy  to  detect  the  Warnings,  Cautions,  and  Advisories  on  the  MFD  and  entry 
into  operational  limits.  Three  pilots  commented  that  the  colors  used  for  the  symbology  (e.g., 
magenta)  on  the  MFDs  made  it  difficult  for  them  to  quickly  and  easily  distinguish  the 
symbology.  The  pilots  reported  that  the  CDS  5  software  was  somewhat  quicker  and  easier  to  use 
than  the  CDS  4  software.  They  also  reported  that  the  6x8  display  enhanced  SA  and  the  NMS 
functionality  is  an  improvement  over  the  MMS  on  the  OH-58D  Kiowa  Warrior. 

4.5  Mission  Success 

All  of  the  missions  performed  by  the  aircrews  were  rated  as  “successful”  by  the  SMEs  who 
observed  each  mission. 

4.6  Simulator  Sickness 

The  pilots  and  copilots  reported  that  they  typically  experienced  very  mild  to  mild  simulator 
sickness  symptoms  during  the  evaluation.  The  OH-58F  simulator  did  not  induce  debilitating 
simulator  sickness  symptoms  and  should  continue  to  be  a  suitable  simulation  environment  for 
future  assessments. 

4.7  Audio  Alerting 

The  overall  design  of  the  audio  alerting  system  received  good  to  high-ratings  from  the  pilots  for 
alert  presentation  logic,  the  utility  of  the  audio  alerting  system  functions  and  characteristics,  the 
audio  workload  associated  with  using  the  system,  and  pilots’  reported  air  vehicle  and  tactical  SA 
achieved  during  the  simulated  missions. 
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4.8  Simulator  Functionality 


Simulator  functionality  was  somewhat  limited  during  the  HFE  #2  assessment.  The  information 
and  data  listed  in  the  Results  and  Summary  sections  of  this  report  should  be  interpreted  based  on 
these  limitations.  Future  simulations  should  include  full  functionality  of  the  simulated 
subsystems  and  crewstation  interface. 


5.  Recommendations 


The  following  recommendations  are  made  to  enhance  the  overall  effectiveness  and  suitability  of 
the  OH-58F  crewstation  development  and  assessment  process: 

•  Continue  to  upgrade  the  OH-58F  simulator  to  make  it  as  representative  of  the  production 
design  as  possible. 

•  Continue  to  maximize  the  amount  of  OH-58F  crewstation  experience  for  pilots  prior  to 
future  assessments. 

•  Workload,  SA,  crewstation  interface  and  simulator  sickness  (as  applicable)  data  should  be 
collected  during  future  OH-58F  crewstation  assessments,  developmental  testing,  and 
operational  testing.  This  continuity  will  allow  direct  comparison  (after  further 
development  and  integration  of  the  OH-58F  crewstation)  to  assess  workload,  SA,  and  the 
crewstation  interface. 

•  Address  and  resolve  the  PCI  issues  identified  during  the  HFE  simulation  assessment. 
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Appendix  A.  Bedford  Workload  Rating  Scale  Scores  and  Pilot  Comments 


This  appendix  is  presented  in  its  original  form  without  editorial  change. 


BEDFORD  WORKLOAD  RATING  SCALE  (BWRS) 


Workload  Description  “Rating” 


Was  workload 
satisfactory 
without  reduction  in 
spare  (workload)  capacity? 


Was  workload  tolerable 
for  the  task? 


NO 


Was  it  possible  to 
complete  the  task? 


Pilot  Decisions 


Workload  insignificant 

1 

Workload  low 

2 

Enough  spare  capacity  for  all 
desirable  additional  tasks 

3 

Task  abandoned.  Pilot  unable  to 
apply  sufficient  effort 


Insufficient  spare  capacity  for  easy 
attention  to  additional  tasks 

4 

Reduced  spare  capacity.  Additional 
tasks  camiot  be  given  the  desired 
amount  of  attention 

5 

Little  spare  capacity:  level  of  effort 
allows  little  attention  to  additional 
tasks 

6 

Very  little  spare  capacity,  but 
maintenance  of  effort  in  the  primary 
tasks  not  in  question 

7 

Very  high  workload  with  almost  no 
spare  capacity.  Difficulty  in 
maintaining  level  of  effort 

8 

Extremely  high  workload.  No  spare 
capacity.  Serious  doubts  as  to  ability 
to  maintain  level  of  effort 

9 

10 
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BEDFORD  WORKLOAD  RATING  SCORES 


Task 

Flight  and  Mission  Tasks 

No. 

1026 

Maintain  Airspace  Surveillance 

1028 

Perform  Hover  Power  Check 

1030 

Perform  Hover  Out-Of-Ground-Effect  (OGE)  Check 

1032 

Perform  Radio  Communication  Procedures 

1038 

Perform  Hovering  Flight 

1040 

Perform  Visual  Meteorological  Conditions  (VMC) 

Takeoff 

1044 

Navigate  by  Pilotage  and  Dead  Reckoning 

1046 

Perform  Electronically  Aided  Navigation 

1048 

Perform  Fuel  Management  Procedures 

1052 

Perform  VMC  Flight  Maneuvers 

1058 

Perform  VMC  Approach 

1066 

Perform  A  Running  Landing 

1070 

Respond  to  Emergencies 

1074 

Respond  to  Engine  Failure  in  Cruise  Flight 

1140 

Perform  Nose  Mounted  Sensor  (NMS)  Operations 

1142 

Perform  Digital  Communications 

1155 

Negotiate  Wire  Obstacles 

1170 

Perform  Instrument  Takeoff 

1176 

Perform  Non  Precision  Approach  (GCA) 

1178 

Perform  Precision  Approach  (GCA) 

1180 

Perform  Emergency  GPS  Recovery  Procedure 

1082 

Perform  an  Autorotation 

1182 

Perform  Unusual  Attitude  Recovery 

Pilot  Workload 
Rating 


Copilot 

Workload 

Rating 


2.00 

2.50 

2.86 

1.83 

2.00 

2.00 

2.00 

2.00 

3.00 

3.86 

2.29 

2.00 

3.00 


6.00 

5.00 

6.50 


4.00 

5.00 


Operate  ASE/transponder 


Respond  to  IMC  Conditions 


Perform  Refueling  /  Rearming  Operations 


Perform  Electronic  Countermeasures  /  Electronic 
Counter-Countermeasures 


Transmit  Tactical  Reports 


Perform  Terrain  Flight  Takeoff 


Perform  Terrain  Flight 


Perform  Terrain  Flight  Approach 


Perform  Masking  and  Unmasking 


Perform  Terrain  Flight  Deceleration 


Perform  Actions  on  Contact 


Perform  Weapons  Initialization  Procedures 


Perform  Firing  Techniques 


Engage  Target  with  .50  Cal 


Engage  Target  with  Hellfire 


Engage  Target  with  Rockets 


Perform  Aerial  Observation 


Perform  Target  Handover 


Aerial  Observation 


Call  for  Indirect  Fire 


Perform  Multi -Aircraft  Operations 


Perform  Combat  Maneuvering  Flight 


Perform  Close  Combat  Attack 


Perform  Combat  Position  Operations 


Call  for  Tactical  Air  Strike 


Zone  Reconnaissance 


Route  Reconnaissance 


Area  Reconnaissance 


Aerial  Surveillance 


Overall  Workload  for  the  Mission 
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Pilot  Workload  Comments: 


Comments  for  flight  and  mission  tasks  that  were  given  a  workload  rating  of  ‘5’  or  higher  during 
missions: 

Task  (1028)  Perform  Hover  Power  Check 

•  Different  fidelity  in  simulator  than  aircraft. 

Task  (1038)  Perform  Hovering  Flight 

•  Hovering  difficult  due  to  simulator  and  depth  perception. 

•  Different  fidelity  in  simulator  than  aircraft. 

Task  (1182)  Perform  Unusual  Attitude  Recovery 

•  Focus  is  on  recovering  aircraft  so  everything  else  is  mute. 

Task  (1184)  Respond  to  IMC  Conditions 

•  Aircraft  is  the  focus  so  other  tasks  are  dropped. 

Task  (1140)  Perform  Nose  Mounted  Sensor  (NMS)  Operations 

•  Attention  focused  inside  for  almost  the  duration.  NMS  ops  took  up  so  much  time  that 
ASE,  airstrike,  reports,  etc.  quality  suffered. 

Task  (1170)  Perform  Instrument  Takeoff 

•  Higher  workload  trying  to  control  aircraft  IIMC.  Partly  due  to  the  nature  of  the  task, 
compounded  by  lack  of  “seat  of  the  pants”  feel  and  control  input  lag  time  of  simulator. 

Task  (1180)  Perform  Emergency  GPS  Recovery  Procedure 

•  Higher  workload  trying  to  control  aircraft  IIMC.  Partly  due  to  the  nature  of  the  task, 
compounded  by  lack  of  “seat  of  the  pants”  feel  and  control  input  lag  time  of  simulator. 

•  Simulator  flight  controls  are  not  as  responsive  as  the  real  aircraft.  I  feel  like  I  was 
constantly  over  correcting  for  inputs. 

Task  (1074)  Respond  to  Engine  Failure  in  Cruise  Flight 

•  Didn’t  have  audio  indicators  of  engine  failure 

Task  (1082)  Perform  an  Autorotation 

•  Engine  Out  at  140ft  with  50deg  left  bank. 

•  Didn’t  have  audio  indicators  of  engine  failure 

•  Different  fidelity  in  simulator  than  aircraft. 

Miscellaneous  Workload  Comments: 

•  IMC/Unusual  Attitude/ Airspace  Surveillance/NMC/Overall  Workload  -  very  busy  cockpit 
makes  things  difficult  to  find. 

•  Operating  the  NMS  required  additional  attention  due  to  the  newness  of  it  and  trying  to 
adjust  it  with  switches  that  do  not  work  (simulator  issue). 
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SME  Workload  Comments: 


•  Mission  #8  -  Workload  during  engagements  and  operations  vicinity  of  the  objective  high 
enough  to  cause  left  seat  pilot  to  momentarily  lose  SA  causing  disorientation/association  of 
ASE  indications,  miss  some  ASE  indications  completely  and  also  some  radio  calls. 

•  Mission  #8  -  When  in  contact,  left  seat  pilot  lost  SA  and  missed  some  ASE  indications  as 
well  as  some  radio  calls. 

•  Mission  #4  -  Left  seat  pilot  had  RFD  issues  as  well  as  SA  while  attempting  NMS 
operations  to  ID  targets.  RFD  issues  were  failing  to  ID  what  radio  he  was  talking  on. 

•  Mission  #2  -  Workload  was  increased  due  to  some  fixation  from  NMS  simulation. 

•  Mission  #3  -  Right  seat  pilot  transferred  controls  during  IIMC  recovery  ops. 
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Appendix  B.  Situation  Awareness  Ratings  and  Comments 


This  appendix  is  presented  in  its  original  form  without  editorial  change. 
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China  Lake  Situational  Awareness  Rating  Scale  (CLSA) 


VERY  GOOD 
1 


GOOD 

2 


Copilot  Mean - 

Rating  _ 

2.75 

(SD  =  .886)  EQUATE 

r  3 


Full  knowledge  of  aircraft  energy  state,  tactical  environment  and 
mission 

Full  ability  to  anticipate  and  accommodate  trends 


Full  knowledge  of  aircraft  energy  state,  tactical  environment  and 
mission 

Partial  ability  to  anticipate  and  accommodate  trends 
No  task  shedding 

- < - 

Full  knowledge  of  aircraft  energy  state,  tactical  environment  and 
mission 

Saturated  ability  to  anticipate  and  accommodate  trends 
Some  shedding  of  minor  tasks 


Pilot  Mean 
Rating 
2.50 

(SD  =  .535) 


POOR 

4 

•  Fair  knowledge  of  aircraft  energy  state,  tactical  environment  and 
mission 

•  Saturated  ability  to  anticipate  and  accommodate  trends 

•  Shedding  of  all  minor  tasks  as  well  as  many  not  essential  to  flight 
safety  and  mission  effectiveness 

VERY  POOR 

5 

•  Minimal  knowledge  of  aircraft  energy  state,  tactical  environment  and 
mission 

•  Oversaturated  ability  to  anticipate  and  accommodate  trends 

•  Shedding  of  all  tasks  not  absolutely  essential  to  flight  safety  and 
mission  effectiveness 
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Battlefield 

Elements 

Very  High- 
Level  of 
Situation 
Awareness 

Fairly  High- 
Level  of 
Situation 
Awareness 

Borderline 

Fairly  Low 
Level  of 
Situation 
Awareness 

Very  Low 
Level  of 
Situation 
Awareness 

Right 

Seat 

Left 

Seat 

Right 

Seat 

Left 

Seat 

Right 

Seat 

Left 

Seat 

Right 

Seat 

Left 

Seat 

Right 

Seat 

Left 

Seat 

Location  of 
Enemy  Units 

0  % 

0  % 

63  % 

37  % 

37  % 

63  % 

0  % 

0  % 

0  % 

0  % 

Location  of 
Friendly  Units 

0  % 

0  % 

88  % 

75  % 

13  % 

13  % 

0  % 

13  % 

0  % 

0  % 

Location  of  My 
Aircraft  During 
Missions 

13  % 

13  % 

50  % 

50  % 

25  % 

0  % 

13  % 

38  % 

0  % 

0  % 

Location  of 
Cultural 
Features  (e.g., 
bridges) 

0  % 

0  % 

50  % 

40  % 

50  % 

40  % 

0  % 

20  % 

0  % 

0  % 

Route 

Information 
(ACPs,  BPs, 
EAs,  RPs,  etc.) 

0  % 

0  % 

88  % 

75  % 

13  % 

13  % 

0  % 

13  % 

0  % 

0  % 

Status  of  My 
Aircraft  Systems 
(e.g.,  fuel 
consumption) 

0  % 

0  % 

75  % 

71  % 

25  % 

14  % 

0  % 

14  % 

0  % 

0  % 

Pilot  Situational  Awareness  Comments 

SA1:  If  you  rated  ‘Poor’  or  ‘Very  Poor’  -  explain. 

•  Would  have  been  ‘Good’  for  ‘Very  Good’  except  for  initial  trouble  with  associating  target 
position  in  relationship  to  town  when  using  NMS  orientation,  etc. 

•  Difficult  to  determine  place  on  the  ground  in  relation  to  objective,  friendlies,  enemy,  LZ, 
PZ/Cardinal  Direction. 

SA2:  If  you  rated  a  battlefield  element  ‘Fairly  Low’  or  ‘Very  Low’  -  explain. 

•  At  times  I  went  through  boundaries.  I  think  it  was  due  to  phase  line  colors  (hard  to  see  on 
display)  and  map  background.  (Lack  of  experience). 

•  Location  of  aircraft  during  mission  due  to  lack  of  experience  with  NMS  -  unfamiliar  with 
NMS  switches. 

•  Map  color  on  5x7  and  6x9  in  addition  to  character  colors  made  things  difficult  to  find  on 
the  screen.  Colors  were  a  little  too  much  to  process. 

•  Fuel  consumption/ ACFT  Systems  were  difficult  to  find. 
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My  attention  while  in  the  left  seat  is  focused  so  intently  inside  that  it  was  extremely 
difficult  to  come  back  outside  and  figure  out  where  or  which  way  was  North. 


Miscellaneous  comments: 

•  Simulator  display  issue  location  of  enemy  units  -  ‘Good’  in  town,  ‘Fair’  with  personnel  in 
open. 

SME  Situation  Awareness  Comments: 

•  Mission  #6  -  Crew  maintained  very  good  SA  throughout  entire  mission. 

•  Mission  #8  -  Crew  did  not  exploit  NMS  capabilities.  Consistently  positioned  aircraft  too 
close  to  objective,  leading  to  multiple  enemy  engagements.  When  all  crew  Hellfire 
expended,  crew  elected  to  engage  T-72s,  ZSU-23-4,  and  BMP  with  .50  cal  instead  of 
engagements  with  CAS,  artillery,  or  ground  forces. 

•  Mission  #8  -  Crews  were  focused  on  OBJ  Hawk  and  did  not  realize  additional  enemy 
targets  around  outside  perimeter  of  OBJ  Hawk.  Poor  crew  decision  making  and  lack  of 
proper  TTP’s  had  negative  effect  on  success  of  mission. 

•  Mission  #1  -  Crews  were  given  “help”  to  see  truck  under  wires,  which  led  to  a  lack  of  SA, 
but  as  mission  progressed  crew  gained  SA. 

•  Mission  #2  -  Simulation  caused  some  SA  issues.  Crew  also  had  some  NMS  fixation  during 
recon  of  route  and  area  around  cordon  and  search  objective. 

•  Mission  #3  -  First  mission  for  this  crew  -  SA  increased  as  familiarity  increased.  By  end  of 
mission,  crew  had  good  SA. 

•  Mission  #3  -  Crew  maintained  NMS  in  a  zoomed  mode  and  missed  some  SA  due  to  that  for 
zoomed  in.  Aircrew  also  had  battlefield  graphics  color  issues  which  reduced  SA  and  crew 
flew  past  a  phase  line  restriction  boundary. 

•  Mission  #4  -  Took  awhile  for  left  seater  to  get  accustomed  to  NMS  symbology. 

•  Mission  #4  -  Crew  had  some  SA  issues  most  likely  with  battlefield  graphics  which  caused 
them  to  move  across  PL  New  Mexico  restriction.  Also  had  SA  issues  while  attempting  to 
ID  PAX  at  NAI  2.  Sim  restricts  actual  visual  recognition  of  obvious  sightings. 

•  Mission  #5  -  None.  Excellent  use  of  all  assets  (JVMG,  BFT,  NMS). 

•  Mission  #5  -  Crew  started  to  increase  workload  which  caused  reduction  in  SA  and  caused 
boundary  restriction  violations  a  couple  of  times. 
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Appendix  C.  Pilot-Crewstation  Interface  (PCI)  Ratings  and  Comments 


This  appendix  is  presented  in  its  original  form  without  editorial  change. 


PV1.  The  following  table  lists  the  components  (e.g.,  display  pages,  sub-pages,  overlays)  of 
the  KW  CASUP  crewstation.  For  each  component,  indicate  whether  or  not  you  experienced 
a  problem  using  the  component  in  a  quick  and  efficient  manner  during  the  missions.  Check 
‘Yes’  if  you  experienced  one  or  more  problems.  Check  ‘No’  if  you  did  not  experience  any 
problems.  Check  ‘Not  Used’  if  you  did  not  use  the  component  during  the  simulation. 


•  Multifunction  Displays  (MFD) 


o 

Vertical  Situation  Display 

Yes  0% 

No  100% 

o 

Horizontal  Situation  Display 

Yes  0% 

No  100% 

o 

NMS  Operations 

Yes  0% 

No  100% 

o 

COM  Pages 

Yes  0% 

No  100% 

o 

NAV  Pages 

Yes  0% 

No  100% 

o 

Digital  Map 

Yes  0% 

No  100% 

o 

Engine  Instruments 

Yes  25% 

No  75% 

o 

ASE  Pages 

Yes  0% 

No  100% 

o 

Weapons  Pages 

Yes  0% 

No  100% 

o 

Managing  GPS/Flight  Plan 

Yes  0% 

No  100% 

o 

Mission  Page 

Yes  0% 

No  75% 

Not  Used  25% 

o 

Warning,  Caution,  Advisory  Disp.  Yes  0% 

No  100% 

o 

‘Direct  To’ 

Yes  0% 

No  100% 

o 

ACP  Function 

Yes  0% 

No  100% 

o 

Fuel  Management  Pages 

Yes  25% 

No  75% 

o 

Data  Function 

Yes  0% 

No  100% 

o 

ACP  Function 

Yes  0% 

No  100% 

o 

Autopaging 

Yes  0% 

No  100% 

you  answered  “Yes”  to  any  of  the  questions,  describe  a)  the  problems  you  experienced, 

)  how  much  the  problems  degraded  your  performance,  and  c)  any  recommendations  you 

have  for  improving  the  design  of  the  components. 
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Pilot  comments: 


•  All  the  systems  were  very  easy  to  use.  This  is  a  good  improvement  over  current 
software.  I  was  slower  at  tasks  in  the  simulator,  but  that  was  due  to  the  newness  of  the 
system. 

•  Engine  instruments  are  hard  to  find.  Especially  during  emergencies. 

•  Fuel  management  page  timer  should  be  working  to  accurately  reflect  realism.  It  would 
create  an  extra  advisory  for  pilots  to  respond  to.  (Sim  issue). 

PV2.  How  quickly  were  you  able  to  navigate  through  the  pages,  sub-pages  and/or  overlays 
for: 


Vertical  Situation  Display  (Circle  one) 


■\  Avg.  Rating 
1.75 


2 


3 


4  5 


Very  y 

Somewhat 

Borderline 

Somewhat 

Very 

Quickly 

Quickly 

Slowly 

Slowly 

Horizontal  Situation  Display  (Circle  one) 

1 

Avg.  Rating 

2 

3 

4 

5 

Very 

SonTewhat 

Borderline 

Somewhat 

Very 

Quickly 

Quickly 

Slowly 

Slowly 

Digital  Map 

(Circle  one) 

1 

Avg.  Rating 

2 

3 

4 

5 

Very 

Somewhat 

Borderline 

Somewhat 

Very 

Quickly 

Quickly 

Slowly 

Slowly 

Engine  Instruments 

(Circle  one) 

1 

2 

Avg.  Rating 
2.25 

3 

4 

5 

Very  Somewhat  Borderline  Somewhat  Very 

Quickly  Quickly  Slowly  Slowly 


If  you  answered  ‘Somewhat  Slowly’,  or  ‘Very  Slowly’  to  any  of  the  questions,  list  the 
component  and  why  navigation  was  slow  (e.g.,  ‘navigating  the  menu  system  on  the  digital 
map  was  a  slow  process  due  to  having  to  page  through  several  screen  displays’). 
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Pilot  comments: 


•  No  comments. 

PV3.  Please  answer  the  following  questions  regarding  the  Nose  Mounted  Sensor  (NMS). 
PV3-1.  Did  you  experience  any  problems  using  the  following  NMS  switches/controls? 


o  Sensor  Select 

Yes  0% 

No  100% 

o  FOV  Select 

Yes  0% 

No  100% 

o  Laser 

Yes  0% 

No  100% 

o  LOS  Designate 

Yes  0% 

No  100% 

o  Range/Polarity 

Yes  0% 

No  100% 

o  Manual  Slave 

Yes  25% 

No  75% 

o  Point  Track 

Yes  0% 

No  100% 

If  you  answered  “Yes”  to  any  of  the  questions,  describe  a)  the  problems  you  experienced, 
b)  how  much  the  problems  degraded  your  performance,  and  c)  any  recommendations 
you  have  for  improving  the  design  of  the  NMS  switches/controls. 

Pilot  comments: 

•  NMS  doesn’t  move  in  forward  mode.  Can  it  move  as  it  does  now? 

PV4.  Did  you  have  difficulty  using  any  of  the  switches  on  the  collective  or  cyclic  grips? 


Collective  Grip 

Yes  0% 

No  100% 

Cyclic  Grip 

Yes  0% 

No  100% 

If  you  answered  “Yes”  for  either  flight  control,  please  list  which  flight  control  and  switch(es), 
and  the  problems  you  experienced  (e.g.,  confused  two  switches  due  to  similar  shape, 
switch  was  too  hard  to  reach). 

Pilot  comments: 

•  No  comments 


PV5.  Was  there  any  symbology  depicted  on  the  following  displays/pages  that  was  difficult 
to  quickly  and  easily  understand,  cluttered,  or  otherwise  difficult  to  use? 
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Vertical  Situation  Display 

Yes  0% 

No  100% 

Horizontal  Situation  Display 

Yes  25% 

No  75% 

ESIS 

Yes  50% 

No  50% 

Engine  Instruments 

Yes  25% 

No  75% 

Digital  Map 

Yes  50% 

No  50% 

ASE 

Yes  25% 

No  75% 

NMS  Pages 

Yes  25% 

No  75% 

If  you  answered  “Yes”  to  any  of  the  questions,  please  describe  a)  the  display/page,  b)  the 
symbology  that  was  difficult  to  understand,  c)  how  the  symbology  degraded  your 
performance,  and  d)  any  recommendations  you  have  for  improving  the  design  of  the  display 
page  and/or  symbology. 

Pilot  comments: 

•  ESIS  is  too  small.  Engine  instruments  are  too  new  for  me  to  interpret. 

•  ESIS:  The  low  position  of  this  instrument  makes  it  hard  to  read. 

•  HSD/MAP  pages  are  extremely  cluttered.  NMS  features  make  navigation  through  the 
system  difficult. 

•  Digital  Map:  Magenta  for  NMS  is  difficult  to  see. 

•  ASE:  Blue  tracking  is  difficult  on  black  background. 


PV6.  How  easy  was  it  to  detect  the  following  indication  on  the  displays? 


Warning/Caution/Advisory  (MFD) 


1 


Avg.  Rating 
1.5 


2 


Very  T  Somewhat 
Easy  Easy 


3 


Borderline 


4 


Somewhat 

Difficult 


5 


Very 

Difficult 


Entry  into  Operational  Limits 


1 


Avg.  Rating 
1.5 


2 


Very  v  Somewhat 
Easy  Easy 


3 


Borderline 


4 


Somewhat 

Difficult 


5 


Very 

Difficult 
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Low  Fuel 


Avg.  Rating 


VeTry  Somewhat 

Easy  Easy 


3 


4 


Borderline  Somewhat 
Difficult 


5 


Very 

Difficult 


If  you  answered  “Somewhat  Difficult”,  or  “Very  Difficult”,  list  which  indication  you  had 
difficulty  detecting/understanding,  why  you  had  difficulty,  and  any  recommendations 
to  make  the  indication  more  easily  detectable  and/or  understandable. 

Pilot  comments: 

•  No  comments. 

PV7.  Did  you  have  any  problems  using  the  overhead  panels  due  to  location,  inaccurate 
labeling,  etc? 

Yes  0%  No  50%  NU  50% 

PV8.  Did  you  have  any  problems  using  the  following  switches  and  controls  on  the 
instrument  panel? 


CMWS  Manual  Dispense  Switch 

Yes  0% 

No  75% 

NU  25% 

SCAS  Control  Panel 

Yes  0% 

No  50% 

NU  50% 

FADEC  Switch 

Yes  0% 

No  25% 

NU  75% 

MFK 

Yes  0% 

No  100% 

Armament  Control  Panel 

Yes  0% 

No  100% 

Channel  Select  Switch 

Yes  0% 

No  100% 

If  yes,  list  the  switches  and/or  controls  and  describe  the  problem  (s) 

Pilot  comments: 

•  No  comments. 

PV9a.  Did  you  have  any  problems  viewing  information  (symbology,  text,  etc)  on  the 
following  displays: 

6x8  Display  Yes  50%  No  50% 
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5x7  Display 
ESIS  Display 


Yes  25% 


No  75% 


Yes  50%  No  50% 

If  yes,  list  the  information  you  had  problems  viewing: 

Pilot  comments: 

•  Color  MFDs  can  make  things  difficult  to  find. 

•  NMS  magenta  difficult  to  see. 

•  ESIS:  Hard  to  read  with  low  mounting  position. 

PV9b.  Did  you  have  any  problems  using  the  bezels,  knobs  or  hot  keys  on  the  6x8 
display? 

Yes  0%  No  100% 

PV10.  Did  the  LPCAP  (digital  ICS)  restrict  cyclic  movement  in  the  right  seat? 

Yes  0%  No  100% 

If  yes,  describe  the  how  much  the  LPCAP  restricted  movement  and  in  which  axis: 

No  comments: 


PV11.  Rate  whether  the  CDS5  software  was  much  more  or  less  effective  (quicker  and 
easier  to  use)  than  the  CDS4  software  you  have  used  on  the  KW. 


Avg.  Rating 


Much  More  Somewhat  About  The  Somewhat  Much  Less 
Effective  More  Effective  Same  Less  Effective  Effective 


If  ‘Somewhat  Less’  or  ‘Much  Less  Effective’,  explain  why: 
Pilot  comments: 

•  No  comments. 
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Intentionally  Left  Blank. 
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Appendix  D.  Top  Crewstation  Improvements 


This  appendix  is  presented  in  its  original  form  without  editorial  change. 
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OH-58F  Functionality  That  Is  An  Improvement  vs.  OH-58D  Kiowa  Warrior 
Nose  Mounted  Sensor 

•  NMS  optics  and  functions  is  an  improvement  over  current  aircraft 

•  NMS  is  outstanding. 

NMS  is  great  for  increased  stand-off 

Displays 

•  The  6x8  is  great  and  allows  for  additional  situational  awareness  for  the  pilot  on  the 
controls 

•  6x8  MFD  is  an  improvement  (improves  situational  awareness) 

•  Systems  are  organized  well  on  the  6x8  and  using  PWR  is  nice  to  have  versus  TGT  or 
highest  value  indicated. 

•  Color  MFDs  are  an  improvement 

•  RFD  is  much  more  useful. 

•  Auto  paging  is  an  improvement 

•  Engine/aircraft  performance  displays  are  an  improvement 


Improvements  that  need  to  be  made  to  OH-58F  crewstation: 

•  Digital  map  (magenta)  is  difficult  to  read 

•  ASE  blue  tracking  on  black  background  is  difficult  to  see  -  needs  improvement. 

•  Declutter  options  -  option  to  remove  bezel  labels  but  remain  functional. 

•  Add  a  compass  rose  to  the  6x8. 

•  Make  the  ESIS  easier  to  read. 

•  Consider  relocating  fuel  quantity  location 

•  Need  a  NMS  desktop  trainer. 

•  Expanded  checklist  for  NMS  operations. 

•  Label  power  display  with  the  power  monitored. 

•  A  negative  would  be  that  the  left  seater  will  spend  more  time  inside  the  aircraft  (due  to 
improved  NMS). 

•  Need  a  hologram  sight  for  weapons  on  dash. 

•  Need  tones  with  all  WCA’s. 

•  TGT  list  hotkey  from  NMS  page. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AAR 

after  action  review 

AOI 

area(s)  of  interest 

APEX 

Advanced  Prototype  Engineering  and  Experimentation 

ARH 

Armed  Reconnaissance  Helicopter 

ARH 

Armed  Reconnaissance  Helicopter 

ARL 

U.S.  Army  Research  Laboratory 

ASL 

Applied  Science  Laboratories 

ATM 

Aircrew  Training  Manual 

BHIVE 

Battlefield  Highly  Immersive  Visual  Environment 

BIT 

Built-In  Test 

BWRS 

Bedford  Workload  Rating  Scale 

CDD 

Capability  Development  Document 

CDS 

Control  Display  Subsystem 

CIB 

Controlled  Image  Base 

CLSA 

China  Lake  Situational  Awareness 

CPC 

Comanche  Portable  Cockpit 

DAMA 

Demand  Assigned  Multiple  Access 

DIS 

Distributed  Interactive  Simulation 

DREN 

Defense  Research  and  Engineering  Network 

EDS 

Engineering  Development  Simulator 

EUD 

Early  User  Demo 

GPS 

Global  Positioning  System 

HFE 

human  factors  engineering 

HLA 

High-Level  Architecture 
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HOG 


HRED 

HUMS 

IMC 

JVMF 

KW  CASUP 

LEUE 

LOBL 

MFD 

MMS 

NDI 

NMS 

OneSAF 

OTW 

PCI 

PIM 

PLRA 

SA 

SME 

SSDD 

SSQ 

TCMRA 

TRADOC 

TS 

TTP 

VFR 


Hands-On  Grip 

Human  Research  and  Engineering  Directorate 

Health  Usage  Monitoring  System 

instrument  meteorological  conditions 

Joint  Variable  Message  Format 

Kiowa  Warrior  Cockpit  and  Sensor  Upgrade  Program 

Limited  Early  User  Evaluation 

lock-on-before-launch 

multifunction  display 

Mast  Mounted  Site 

Northern  Digital  Incorporated 

Nose  Mounted  Sensor 

One  Semi-Automated  Forces 

out-the-window 

Pilot-Crewstation  Interface 

Pulse  Interval  Module 

Psycho-Linguistic  Research  Associates 

situational  awareness 

subject  matter  expert 

Systems  Simulation  and  Development  Directorate 
Simulator  Sickness  Questionnaire 
TRADOC  Capability  Manager,  Reconnaissance  Attack 
Training  and  Doctrine  Command 
Total  Severity 

tactics,  techniques,  and  procedures 
visual  flight  rules 
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VMC 

VTR 


visual  meteorological  conditions 
Video  Tape  Recorder 
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NO.  OF 

COPIES  ORGANIZATION 

1  ADMNSTR 
(PDF)  DEFNS  TECHL  INFO  CTR 
ATTN  DTIC  OCP 

8725  JOHN  J  KINGMAN  RD  STE  0944 
FT  BELVOIR  VA  22060-6218 

1  HC  US  ARMY  RSRCH  LAB 
ATTN  IMALHRA 
MAIL  &  RECORDS  MGMT 
ADELPHI  MD  20783-1197 

1  US  ARMY  RSRCH  LAB 
(PDF)  ATTN  RDRL  CIO  LL  TECHL  LIB 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 

(PDF)  RDRL  HRM  C  A  DAVISON 
320  MANSCEN  LOOP  STE  1 15 
FORT  LEONARD  WOOD  MO  65473 

1  ARMY  RSCH  LABORATORY  -  HRED 

(PDF)  RDRL  HRM  D 
T  DAVIS 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

1  ARMY  RSCH  LABORATORY  -  HRED 

(HC)  RDRL  HRS  EA  DR  V  J  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 

FORT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 

(PDF)  RDRL  HRM  DG  J  RUBINSTEIN 
BLDG  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

(PDF)  ARMC  FIELD  ELEMENT 

RDRL  HRM  CH  C  BURNS 
THIRD  AVE  BLDG  1467B  RM  336 
FORT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 

(PDF)  AWC  FIELD  ELEMENT 

RDRL  HRM  DJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FORT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

(PDF)  RDRL  HRM  CK  J  REINHART 
10125  KINGMAN  RD  BLDG  317 
FORT  BELVOIR  VA  22060-5828 


1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  AY  M  BARNES 
2520  HEALY  AVE 
STE  1172  BLDG  51005 
FORT  HUACHUCA  AZ  85613-7069 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  AP  D  UNGV ARSKY 
POPE  HALL  BLDG  470 
BCBL  806  HARRISON  DR 
FORT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  AT  J  CHEN 
12423  RESEARCH  PKWY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  AT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 
(HC)  RDRL  HRM  CU  B  LUT AS -SPENCER 
6501  E  11  MILE  RD  MS  284 
BLDG200A  2ND  FL  RM2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  FIRES  CTR  OF  EXCELLENCE 
FIELD  ELEMENT 
RDRL  HRM  AF  C  HERNANDEZ 
3040  NW  AUSTIN  RD  RM  221 
FORT  SILL  OK  73503-9043 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  AV  W  CULBERTSON 
91012  STATION  AVE 
FORT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
(HC)  HUMAN  RSRCH  AND  ENGRNG 

DIRCTRT  MCOE  FIELD  ELEMENT 
RDRL  HRM  DW  C  CARSTENS 
6450  WAY  ST 
BLDG  2839  RM  310 
FORT  BENNING  GA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  RDRL  HRM  A  MARES 

1733  PLEASONTON  ROAD,  BOX  3 
FORT  BLISS  TX  79916-6816 
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NO.  OF 

COPIES  ORGANIZATION 


8  ARMY  RSCH  LABORATORY  -  HRED 
(5  PDF,  SIMULATION  &  TRAINING 
3  HC)  TECHNOLOGY  CENTER 

RDRLHRT  COL  M  CLARKE  (HC) 
RDRLHRT  I  MARTINEZ  (PDF) 
RDRLHRT  R  SOTTILARE  (HC) 
RDRL  HRT  B  N  FINKELSTEIN  (HC) 
RDRL  HRT  G  A  RODRIGUEZ  (PDF) 
RDRL  HRT  I  J  HART  (PDF) 

RDRL  HRT  M  C  METEVIER  (PDF) 
RDRL  HRT  S  B  PETTIT  (PDF) 

12423  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
(PDF)  (PDF)  HQ  USASOC 

RDRL  HRM  CN  R  SPENCER 
BLDG  E2929  DESERT  STORM  DRIVE 
FORT  BRAGG  NC  28310 

1  ARMY  G1 
(CD)  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  RM  2C489 
WASHINGTON  DC  20310-0300 


ABERDEEN  PROVING  GROUND 


12  DIR  USARL 
(11  PDF,  RDRL  HR 
1  HC)  L  ALLENDER  (PDF) 

P  FRANASZCZUK  (PDF) 

C  COSENZO  (PDF) 

RDRL  HRS 
J  LOCKETT  (PDF) 

RDRL  HRS  B 
M  LAFIANDRA  (PDF) 

RDRL  HRS  C 
K  MCDOWELL  (PDF) 

RDRL  HRS  D 
B  AMREIN  (PDF) 

RDRL  HRS  E 
D  HEADLEY  (PDF) 

RDRL  HRM 

P  S  A V AGE-KNEPS  HIELD  (PDF) 
RDRL  HRM 
C  PAULILLO  (HC) 

RDRL  HRM  B 
C  SAMMS  (PDF) 

RDRL  HRM  C 
L  GARRETT  (PDF) 
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Intentionally  Left  Blank. 
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